[integrations][bedrock] Apply Bedrock native structured output - #1097
Conversation
The Bedrock connection built its ConverseRequest inline in chat(), and every request-building helper was private, so nothing could assert what the connection actually sends. Move the construction into a package-private buildRequest() and cover it. The extraction is behavior-preserving: the moved block is unchanged, and resolveModel() still runs first so a missing model still fails before anything else happens. Five tests, none of which had an equivalent before: model-id resolution across both the configured default and a per-call override, tool config, the system/conversation split, inference config, and message merging. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
Converse exposes outputConfig, its native structured-output surface, from 2.41.22 onward; 2.41.21 does not have it. Pin the minimum version that carries the surface rather than the latest, to keep the change reviewable. The property is shared, so this also moves the Bedrock embedding model, the OpenSearch vector store and the S3 Vectors vector store. All four modules compile and test unchanged, with no source edits. NOTICE follows the resulting set: 27 AWS entries and 10 netty entries, which move to 4.1.130.Final transitively through netty-nio-client, plus a new entry for utils-lite. Nothing leaves the set. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
Bedrock inherited the base 4-arg chat(), which rejects any non-null output schema, so it was the last Java chat model with no native path. Wire Converse's outputConfig so the provider constrains the response, falling back to prompt engineering when the schema, the derivation or the model cannot support it. Capability is an exact match against the model ids AWS documents as supporting structured output, retried once after stripping a leading inference-profile segment. Support is per model rather than per family, so a prefix match would claim a capability the provider denies for one Qwen model while granting it for eight siblings. Anything unrecognised, ARNs included, answers false and takes the fallback: an ARN carries no model information, and a prompt router does not choose its model until the request runs. The AWS SDK ships no schema generator, so the schema is derived with victools and serialised, JsonSchemaDefinition taking a string rather than a JSON value. JacksonModule keeps @JsonProperty names and drops @JsonIgnore fields; without a required check victools emits no required key at all, which would let an empty document satisfy every schema. additionalProperties is never emitted with a non-false value because Bedrock rejects that, which leaves a map's values undescribed. No collision guard: this connection reads only model, temperature and max_tokens, so there is no channel for a caller to have set outputConfig already. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
github-actions
Bot
added
doc-not-needed
wenjin272
left a comment
wenjin272
left a comment
There was a problem hiding this comment.
Thanks for adding native structured-output support for Bedrock and for documenting the provider-specific constraints. I left two inline comments on schema generation and the current model capability list.
| SchemaGeneratorConfigBuilder configBuilder = | ||
| new SchemaGeneratorConfigBuilder( | ||
| SchemaVersion.DRAFT_2020_12, OptionPreset.PLAIN_JSON) | ||
| .with(new JacksonModule()); |
There was a problem hiding this comment.
Could we align the generated schema with Jackson's enum wire values? The bare JacksonModule emits Java enum names instead of their @JsonProperty or @JsonValue values—for example, IN_PROGRESS instead of in-progress—so a schema-valid response can fail during subsequent ObjectMapper deserialization. Ollama and #1098 use similar schema-generation logic, so this may also be a good opportunity to extract the shared configuration into a chat-models/common component. Would you prefer to introduce that abstraction here or handle the cross-provider migration separately?
There was a problem hiding this comment.
Good catch, thanks. Fixed in 8d76388 by enabling FLATTENED_ENUMS_FROM_JSONPROPERTY and FLATTENED_ENUMS_FROM_JSONVALUE. The new testDerivedSchemaFollowsJacksonEnumValues reads every listed value back with a plain ObjectMapper, for one @JsonProperty enum and one @JsonValue enum. #1098 has the same fix in d233bea, and I'll add it to #1117 too.
For the shared config, I'd lean toward a separate PR once these land. There is no shared chat-model module yet, and the recipes differ a bit per provider (map values, closed objects, $ref handling). Adding it here would also tie #1098 and #1117 to this PR. The follow-up would move Ollama, Gemini, Bedrock and watsonx onto one config, and fix the same enum gap in Ollama on main. Does that plan work for you?
| // | ||
| // A card whose capability table carries the bullet in neither column is undocumented rather | ||
| // than negative, and is absent from this set for that reason. | ||
| private static final Set<String> NATIVE_STRUCTURED_OUTPUT_MODELS = |
There was a problem hiding this comment.
AWS now documents Structured outputs support for Claude Opus 4.6 and lists anthropic.claude-opus-4-6-v1 as its Bedrock model ID, but it is absent from this allowlist. Consequently, both the base ID and its us., eu., au., and global. inference-profile forms are classified as incapable and fall back to prompting. Could we add the base ID and cover the direct and prefixed forms in the capability tests? Reference: AWS Claude Opus 4.6 model card.
There was a problem hiding this comment.
Thanks for the pointer. Added in 3f82ff9. While checking, I went through all the Bedrock model cards and found 28 more models that document structured outputs on bedrock-runtime, including Claude Sonnet 4.6. I added those too, so the list has 41 ids now.
capableModels() covers the direct ids, and the prefix test now runs us., eu., apac., au., jp. and global. against anthropic.claude-opus-4-6-v1.
…red output The allowlist was missing models whose AWS model cards document structured-output support on the bedrock-runtime endpoint, Claude Opus 4.6 and Sonnet 4.6 among them, so those models and their inference profiles took the prompt fallback. Add the 29 missing ids, bringing the list to 41, and mirror the list in the capability test. The inference-profile prefix test now resolves against anthropic.claude-opus-4-6-v1. Three comments that did not match the code are corrected along the way. Generated-by: Claude Code 2.1.270 (Claude Opus 5)
The bare JacksonModule lists enum constants by their Java names, while the ObjectMapper that reads the response back uses the @JsonProperty value or @jsonvalue method result. A response that satisfied the schema could then fail to deserialize, e.g. "IN_PROGRESS" against an enum Jackson reads as "in-progress". Enable FLATTENED_ENUMS_FROM_JSONPROPERTY and FLATTENED_ENUMS_FROM_JSONVALUE. Only the listed enum values change: property names and the required set stay as they were. The new test reads every listed value back with a plain ObjectMapper for one enum of each style, so dropping either option fails it. Generated-by: Claude Code 2.1.270 (Claude Opus 5)
github-actions
Bot
added
doc-not-needed
| // property that @JsonProperty renames or @JsonIgnore drops, and a mapped enum constant, | ||
| // have to appear in the schema as the mapper reads them, or a response that satisfies the | ||
| // schema still fails to deserialize. An enum annotating only some constants falls back to | ||
| // Java names for all of them, so its annotated constants do not read back. The two enum |
There was a problem hiding this comment.
One user-visible limitation remains: for a partially annotated enum, victools falls back to Java names for the whole enum, so the generated schema may differ from the values Jackson accepts. Since the existing structured-output integrations do not yet have user-facing documentation either, this does not need to block this PR. Could we document it consistently once the structured-output work across providers is complete, noting that @JsonProperty should be applied to every enum constant?
There was a problem hiding this comment.
Yes, that works. Once the remaining provider PRs land, I'll add a structured output section to the chat model docs that covers every provider, including that an enum used in an output schema needs @JsonProperty on every constant.
2 participants
Linked issue: #280
Purpose of change
Bedrock was the last Java chat model with no native structured-output path. Asking it for a schema did not fall back quietly, it threw, because the connection never overrode the foundation's hook. This wires Converse's
outputConfigso the provider constrains the response, with the existing prompt-engineered path still handling everything else.Bedrock has no Python connection in this repo, so this is Java-only.
The feature needed an SDK bump.
outputConfigappears onConverseRequestatbedrockruntime2.41.22 and does not exist at 2.41.21, so the pin moves to the first version that has it rather than to the latest. That property is shared with three other modules, so the bump moves them too. They compile and test unchanged, andNOTICEfollows the resulting dependency set.It is three commits: a refactor that adds a seam for testing the request, the bump on its own, then the feature. Each is meant to be readable without the other two.
Two decisions are worth a reviewer's attention.
Which models get the native path. Support on Bedrock is documented per model, not per family, so the check is an exact match against the ids AWS lists, retried once after stripping a leading inference-profile segment such as
us.oreu.. A family-prefix match would be wrong in a way that matters: AWS documents one Qwen model as unsupported while six of its siblings are supported. Anything unrecognised takes the prompt fallback, and that includes ARNs, which carry no model information at all, and prompt routers, which do not pick a model until the request runs. Guessing wrong in that direction costs a fallback; guessing wrong in the other costs a runtime error.How the schema is built. The AWS SDK has no schema generator, unlike the OpenAI and Anthropic ones, so the schema is derived locally the way the Ollama connection does it. A couple of constraints there are easy to get wrong and are covered by tests: without Jackson awareness the generator ignores
@JsonPropertynames, emits@JsonIgnorefields and lists enum constants by their Java names rather than the values Jackson reads back, and without a required check it marks nothing required, which would let an empty document satisfy any schema.Two limitations to be aware of rather than surprised by. Recursive types cannot be used as output schemas, since the generated self-reference is something Bedrock rejects up front. And Bedrock warns that the first request against a new schema can spend up to a few minutes compiling it, which here happens inside an operator, so the first record through a new schema may be far slower than the rest.
Tests
The Bedrock chat-model module goes from 12 tests to 84, all offline.
Most of that is coverage this module simply never had. Nothing previously asserted what the connection actually sends, so tools, system messages, inference config and message merging are all tested for the first time. The rest covers the new behaviour: which model ids are accepted and rejected, that the derived schema keeps Jackson property names and enum values and marks fields required, and that the native path engages and disengages when it should. Tools and a schema on the same request are also covered, since Bedrock allows both.
The three other modules affected by the SDK bump run unchanged, and the full build was run so the packaging step is exercised, since a dependency move is exactly what can break it.
What is not covered: none of this calls Bedrock. Those tests need credentials CI does not have, so the request shape is verified and the service's acceptance of it is not. I am verifying that separately against a real endpoint and will report the results here before this is merged. The open questions are whether
outputConfigis accepted as built, which schema details Bedrock tolerates, and whether an unsupported model errors or quietly ignores the request.API
No public API is added or changed, and no new configuration. The visible difference is that a Bedrock connection given an output schema on a supported model now returns a schema-constrained response instead of throwing.
Documentation
doc-neededdoc-not-neededdoc-includedNothing user-facing changes in configuration, and the repo has no structured-output docs page for any provider today.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code 2.1.259 (Claude Opus 5)